SeqFormer: Sequential Transformer for Video Instance Segmentation

In this work, we present SeqFormer for video instance segmentation. follows the principle of vision transformer that models relationships among frames. Nevertheless, observe a stand-alone query suffices capturing time sequence instances in video, but attention mechanisms shall be done with each frame independently. To achieve this, locates an and aggregates temporal information to learn powerful representation video-level instance, which is used predict mask sequences on dynamically. Instance tracking achieved naturally without branches or post-processing. On YouTube-VIS, achieves 47.4 AP ResNet-50 backbone 49.0 ResNet-101 bells whistles. Such achievement significantly exceeds previous state-of-the-art performance by 4.6 4.4, respectively. addition, integrated recently-proposed Swin transformer, much higher 59.3. We hope could strong baseline fosters future research segmentation, meantime, advances field more robust, accurate, neat model. The code available at https://github.com/wjf5203/SeqFormer .